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Each year, countless faculty members are faced with the task of preparing 
dossiers for.use in promotion/tenure proveedings. At the same time, almost as 
many administrators are charged with thr: responsibility of reviewing these 
materials for the purpose of deciding ‘he extent to which they either do, or 
do not meet the established ''criteria'’ for faculty advancement. Regardless of 
whether or not one is being considered for advancement in any given year, 
almost all faculty members have the'r performance evaluated annually, in one 
sense or another. Faculty evaluation is an ongoing process even if there are 
no systematic means for making the assessments. Almost every activity. in academia 
is, at some level, evaluated. The fact is that we all are constantly evaluating 
ourselves, our soi eauices and our peers. 

What factors are considered in the assessment of faculty performance? What 
evidence is utilized in professorial assessment processes? What weights are 
assigned by faculty to factors considered relevant to teaching, scholarship, and 
service when a decision is pending regarding a faculty member's promotion or 


tenure status? Where should a newly appointed faculty member place the majority 


af his energy if he is intent “oon attaining a timely. promotion? ‘n short, what 


performance counts most heavily in academic circles? 

A recently completed 3-year study entitled ''Faculty Teaching: Models for 
Assessment of Quality'' addressed these questions. The study was conducted within 
the Oregon State System of Higher Educat ton and was supported by a grant’ from 
the Fund for the Ifprovement of Postsecondary Education. 

Only part off the results of this study are summarized in this paper. Gener- 
ally, this repo#t contains system-wide data supplied by over 400 teaching and 
administrativef/faculty who were selected on a random/stratified basis. Al] 


tables and figures differentiate responses from college and university faculty. 
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Specific attention is given to promotion/tenure factors that teaching ana 
administrative faculty generally agree are atther influential, uitaPiventtal or 
ambiguous, Factors against’ which faculty would ‘'prefer'' to have their performance 
judged are also briefly presented. Between 1973 and 1976 a total of four such 
system-wide surveys were conducted. Those readers who desire greater methodologi-~ 
cal detail, more extensive discussions of the implications, or citations to 
related studies should consuit’ the expanded version of this report (Thorne, Scott, 
& Beaird, 1976). 

Methods and Data Sources 


Baseline data were collected early in the 1973-74 academic year from two 7 
samples of teaching faculty (one representing departments within four discipline | . 
areas and one representing cross-sections of institutions stratified by academic 
rank). Attempts were made to collect this baseline data From every department~ 
chairman within the six participating campuses. Nearly 1,200 responses were 
received, representing response rates of 82% for the departmental sample, 36% 
for the stratified institutional sample, and 74% for the system's department 
chairmen. , 

The data reported herein were collected at the end of the 1975-76 academic 
year from all teaching and administrative faculty who responded te the initial, 
baseline survey. The response rate for this follow-up sample was over 75% for 
each participating institution. . 

Exhibit 1] presents the survey instrument that provided the means for faculty 
to communicate their opinions regarding which factors were used in their particu- 
lar settings and how much influence each of the factors exerted on the overall] 
promotion/tenure process. The Faculty Perception Questionnaire (FPQ) consists 
of 34 factors or indicators which may or may not influence the promotion/tenure 


process On any given campus. Each respondent rated those “actors that he or she 
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EXHIBIT 1 : F 
Faculty Perception Questionnaire 
Rating Scale fur Part A 
Very Significant 
Moderate Influence Influence - 
2 3 4 5 6 7 
] | | ! | S 
— = | —=e 
SIS | —————S ——— + — ——— aw ——————— eeeEeE—E—EEE 
Col 3 Col 4 ae Col. $ 
Foreach rated 
factor, circle $ 
one choice to 
show an in: { Check (//) 
crease (1) FACTO the five 
“Hecrease (|) factors you 
or no change would prefer 
(0) in influance: to be most 
since 1973 i a influential 


, 1. Publication in scholarly journals and books 
2. Student ratings of instructor performance 


eA Eh Aaa ae li 
3. Contributions to departmental committees 


Support of departmental policy and goals 


. Assessment of course syllabi and examinations by colleagues 


. Informal and impressionistic appraisal of teaching by colleagues 
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7. Formal and systematic appraisal of teaching by colleagues 
8. Success in attracting grant support for research and scholarship’ 
9. Supervision of theses 
10. Personality traits and general altitude 
11” Consultation record on and ol! campus 
12. Record of service on college /university/OSSHE committees 
13. Academic advising . 

14. Membership in professional organizations 


15 Service (no fee) to local and/or state community” 


16. Supervision of held placements (practicum students, residents or feliows) 


Elected offices in professional organizations 


. Public and/or professional presentations of research/artistic products 


Informal (general, impressionistic) colleague appraisal of research and/or artistic 
work 


Formal (product examination) colleague appraisal of research and or artistic work 


21. Obtaining advanced degree 
Evaluation by department chairman 

Informal (general, impressionistic) colleague appraisal of service work 

Formal (product examination) colleague appraisal of service work 

. Evidence of student learning in courses 
. Election to Faculty Senate 
. Effort to remain current in discipline 
. Credit hour production ~ 
Student demand for course 
. Time in academic rank 
Evaluation by schoo!l/department commitiee 
. Availability to students 


Innovative effort in teaching 


34. Forma! and systematic appraisal of the candidate by peers outside of institution 


/ 
5 i 
perceived as having been influential. in his department's promot ion/tenure 
process, on a scale ranging from ''I'' (used but minor influence) to '"'7'' (signi~ 
ficant influence). They also indicated the’ five factors which they most’ 
"preferred'' to be influential in promotion and tenure decisions. Additional 
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information regarding faculty evaluation was collected in structured interviews 
with each tailors president, vice president, other administrative of ficer (viz., 
deans and department heads) and members of campus-wide promotion review committees, 
when such committees existed. 
Results 
Data for the 34 FPQ promotion/tenure factors are presented for college and 
university faculty in Tables 1 and 2 respectively. The means are based on the 
previously discussed 7-point ke scale. The coefficients of consensus (CC) are 
derived scores which range from 9 to 50 and are based upon the proportion of res- 
pondents froma given group indicating the use, or non-use of a particular indica- 
tor. If in a department all faculty members indicate a factor was used (or not 
used), there would be little reason to question the influence (or lack of influence) 
“of the indicator. However, as the number indicating that a factor was used 
ienienaiee the number indicating that it was not used, the actual influence or the 
indicator becomes less clear. (Opinion splits regarding factor use or non-use of 
50/50, 75-25 and 100/0 would result in CC's of 0,:25, and 50 respectively.) 
Coefficients of consensus may be used fo gain some sense of how consensus can vary 
by setting for each factor. A bow decbhiclent indicates that factor's use is 
clouded by uncertainty; conversely, a high score (above 35 or 40) indicates rela- 
tive. consensus among respondents regarding the factor's use or non-use. The 


certainty score is computed by the following formula: 


; 
ce =[  ~ 50 [x 10 


! 
Where: 
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Number of respondents indicating an indicator or factor 
was not used, and i 
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N = Total number of persons in the unit. | 
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Table 


Mean FPQ Values and Coefficients of Consensus 


for College Faculty (N=138) 


Factor Label 

; and Number 

0 
Publications 
Student Ratings 
Service on Departmental Committees 
Support of Department Policy and Goals 
Colleague Assessments of Syllabi 
Informal Colleague Appraisals/Tea:hing 
Formal Colleague Appraisals/Teaching 
Grant Support for Research 
Supervision of Theses an 
Personality Traits and Attitudes 
Consultation Record on/off Campus 
Service on Institution/System Committees 
Academic Advising x 
Membership inaProfessional Organizations 
Service (no fee) to Community 
Supervision of Field Placements 
Elected Offices in Organizations 
Public Presentations of Products 
Informal Colleague-Appraisals/Research 
Formal Colleague Appraisals/Research 
Obtaining Advance Degree 
Evaluation by Department Chairman 
Informal Colleague Appratisals/Service 
Formal Colleague Appraisals/Service 
Evidence of Student Learning in Courses 
Election to FacuJty Senate 
Effort to Remain Current in Discipline 
Credit Hour Production 
Student Demand for Course 
Time in Academic Rank 
Evaluation by School/Department Committee 
Availability to Students 
Innovative Effort in Teaching 
Formal Peer Appraisals (from off campus) 
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Overall 
Factor 


M 


J 


lean 


yw er 


—Wnu —NN—NKMNNNR— SE 


NNW SHH XN 


NY @WLENAON SLO lFnNWOONN OUW —~ KH ON OW NnNWEUBNNON 


Coefficients of 
Consensus 


(50 = 


high; 0 = low 


31 
39 
33 
13 
3] 
32 
4o 


) 
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Table 2 c 


Mean FPQ Values and Coefficients of Consensus 
for University Faculty (N=289) ‘ 


v, 
Overall Coefficients of 
Factor Label Factor Consensus 
Shr HEMIDSE Mean (50 = high, 0 = low) 

1 Publications 5.5 47 
2 Student Ratings 3.8 bh 
3 Service on Departmental Committees 2.6 39 
4 Support of Department Policy and Goals 1.2 12 > 
5 Colleague Assessments of Syllabi 4 31 
6 Informal Colleague Appraisals/Teaching y ae | 30 
7 Formal Colleague Appraisals/Teaching 1.1 18 
8 Grant Support for Research 3.1 19 
9 Supervision of Theses 1.8 7 
10 Personality Traits and Attitudes 22 VW 
1} Consultation Record on/off Campus _ 1.0 15 
12 Service on Institution/System Committees 2.3 29 
13 Academic Advising (ew 4 
14 Membership in Professional Organizations 1.2 8 
15 Service (no fee) to Community 1.9 15 
16 Supervision of Field Placements 25 32 
17 Elected Offices in Organizations 1.7 9 
18 Public Presentations of Products 3.3 30 
19 Informal Colleague Appraisals/Research 2.6 18 
20 Formal Colleague Appraisals/Research 2.5 6 
21 Obtaining Advance Degree a 2.6 4 
22 Evaluation by Department Chairman 4] 35 
23 Informal Colleague Appraisals/Service 1.4 10 
24 Formal Colleague Appraisals/Service .6 30 
25 Evidence of Student Learning in Courses 9 26 
26 Election to Faculty Senate 6 29 
27 Effort to Remain Current in Discipline . 2.4 13 
28 Credit Hour Production 8 26 
29 Student Demand for Course Td 15 
30 Time in Academic Rank 3.4 27 
31 Evaluation by School/Department Committee 4.0 24 
32 Availability to Students 1.4 0° 12 
33 Innovative Effort in Teaching oe VW 


34 Formal Peer Appraisals (from off campus) 2: Ai 7 
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Uncertainty about the use of an indicator may result from several different 
circumstances. Lack of awareness of procedures or criteria by individual faculty 
members is probably the primary and most obvious source of raisrievinine A 
second possible source may result from department-specific conditions (i.e., 
there may be specific departments in which clear communication about an indicator's 
influence in the ovamat fend tenure process. has not occurred.) A third source 
would.reflect nebulous campus procedures, guidelines and/or ol Tes statements. 
And finally, uncertainty esol reflect some characteritstic of the FPQ item 
itself. . , } 

The purpose of Tables 1 and 2 is to present the most fundamental aggregrate 
data obtained from the 1975-76 survey. The patetider of the tables and figures 


in the paper display survey results that are directly based upon “the information 


* ° 


contained in these two tables. 
Factor Influence Clusters ) 

The certainty score dimension seemed to add an important perspective, to the 
data. The thirty-four FPQ factors were arrayed by CC values and by average 
FPQ values, to see if meaningful clusters of factors might emerge. Figures | 
and 2 show the consequences of these plots for college and university data, 
respectively. Three groups of FPQ factors emerged for.each type of campus. They 
were defined as follows: (a) influential factors (high mean's, high CC's); 

(b) definitely minFlusnttal factors (low mean's, high CC's); and (c) ambiguous 
factors (moderate mean's, low CC's). Factors in each influence category may be 
identified by referring to the appropriate subsequent tables and figures. 
Influential Factors : 
Figures 3 and 4 present the influential FPQ factors for college and-univer- 


sity settings, respectively. The items presented in Figures 3 and 4 correspond 


exactly to those contained within ellipse #1 in Figures 1 and 2. Examination 


of Figures 3 and 4 indicate that sampled college and university faculty recognized 
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Figure |. Plot of FPQ mean values by certainty score 
for all College faculty: All ranks and 
disciplines combined. ~ 4 Pa 
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_ (see Figure 4) 


Amb i guous 
FPQ Factors 
(see Table 5) 
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Ellipse #1 


Ellipse A 


Definitely 
Uninfluential 
FPQ Factors 
(see Table 4) 


Ellipse #2 
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“Figure 2. Plot of FPQ mean values by certainty*score | 


for all University ‘faculty: All ranks and 
disciplines combined. 
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Publications ‘ 
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Figure 3. All Collége FPQ values with means ? 2.0 
and certainty scores > 30: All ranks and 
disciplines combined. 


y, 


0 -1.0 2.0 3.0 4.0 5.06.0 7.0 


Publications . - ° % MT 
Evaluation by Department Chairman Mn 
Evaluation by Sueaiivecaerens Committee . QAnaanuaenannngeganaungey 

. Student Ratings ° ; DURDARORDRGRORAROROORON 

. : Time in Acadentc. Rank a nunuunguuuuaeansnnnay 

a 7 Public Presentations of Products , : pananongensuanoansnnt | 


Informal Colleague Appraisals/Teaching 


’ “~~ 
Service on Departmental Committees y All 


: University 


Service on Institution/System Committees 


‘Eigure 4. All University FPQ values with means > 2.0 
and certainty scores ? 25: All ranks and 
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the overriding importance of evaluations by department chairmen, time in academic 
rank, and studert ratings of instructor performance. Faculty from both settings 
also recognized, though to a lesser degree, an influential role for informal 

and impressionistic appraisals of teaching by colleagues, service on institutional 


. 


committees and cantributions to departmentab committees. Two kinds of evidence 
v 

were unique to college settings: obtaining an advanced degree, and personality 

traits and general attitudes. Three kinds of evidence were unique to university 

settings: publications in scholarly journals and books, evaluations by school/ 

departmental committees and public and/or professional presentations of research 

or artistic products. 

In our initial and follow-up interviews we found that the college executive 
officers emphasized the preeminent importance of high quality teaching and 
scholarship in order to attain timely promotion or tenure. In general, they 
indicated a willingness to accept testimovial¢, especially from department heads, 
as sufficient evidence for the former, while the latter is mainly demonstrated 
through the attainment of a doctorate degree. University executive officers 
exhibived a much greater tendency to emphasize breadth of functioning: high 
quality research and teaching, and high quality university service. Research 
was most likely to be formally reviewed and heavily weighted in decisions in 
university settings, as faculty in both settings readily seemed to perceive. 
Uninfluertial Factors 

The smallest clusters of factors arrayed in Figures 1 and 2 were those 
containing the definitely uninfluential FPQ factors (ellipse #2). These were 
factors that most fachlty ajreed were not used in their promotion/tenure processes: 
this accounts for the high CC's corresponding to these-factors. tabties 3 and 4 
present the definitely uninfluential promotion/tenure factors for college and 


university faculty, respectively. nm our numerous discussions with faculty and 


administrators from each of the participating campuses, the most surprising 
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Table 3 
Definitely Uninfluential Promotion/Tenure Factors © 


In College Settings for Teaching and 
Administrative Faculty Combined 


J 
Mean Rating on 7-point Scale 


; (N= 138) : 


Supervision of Theses 2 


FPQ Factor Name 


Formal Colleague Appraisals/Service ay 4 
Formal Colleague Appraisals/Teaching 4 
Formal Peer Appraisals (from off campus) 

Formal Colleague Appraisals/Research 


Colleague Assessments of Syllabi 


Definitely Uninflyential Promot ion/Tenure Factors’ 


In University Settings for Teaching and 
Administrative Faculty Combined 


Mean Rating on 7-phint Scale : 
(N=289) 


FPQ Factor Name 


Colleague Assessments of Syllabi 
Election to Faculty Senate 

Supervision of Field Placements 

Formal Colleague Appraisals/Service 
Credit Hours Production 

Evidence of Student Learning in Courses 


For a factor to have been included in this table it must have been rated in the 
bottom ten (ranks 25-34) by faculty in at least 3 of the following groupings: 
Assistant Professors, Associate Professors, Full Professors, and Chairmen. It 
also must have had a certainty score > 25 for university data and > 30 for 
college data (see figures | & 2). - 7 


aspect of these groups of uninfluential factors had to do with the presence 
of four factors that at least some faculty believe should be of some influence 
Tn the promotion/tenure process: namely, evidence of student learning in courses, 


assessments of course syllabi and examinations by colleagues, formal (product 


examination) colleague appraisals of research and/or artistic work, and formal 
and systematic appraisals of the candidate by peers outside of the institution. 
Ambiguous Factors 

The largest group of factors in Figures | and 2 were those about which 
faculty did not agree regarding their use or non-use A their promotion/tenure 
processes (ellipse #3). This is the cluster of factors that we have chosen to: 
label as presently ambiguous. As a group, these factors probably acdount for 
much of the ''’noise’’ in present faculty evaluation processes. Neither faculty nor 
administrators seem to be sure about whether or not to consider these factors 


2 


when they evaluate professorial performance. Examination of Table 5 indicates 


that college and university faculty overlap on half of the twenty-six ambiguous 


sources of evidence. Insofar as academic disciplines are concerned, there were 
/ / om not great differences within or between college and university campuses. (Readers 
P ~ a > 
y : ; 
Ae interested in such comparisons should refer to Tables 8 through 11.) The most 
, ‘ Seat t 


uncertainty with regard to present factor influence was found in the Arts & 
Humanities. The greatest agreement amongst faculty regarding factor usage was 
in the Physical/Natural Sciences, regardless of campus type. 
Preferred Factors 
All respondents were requested to indicate which five FPQ factors they 

would most prefer to be judged by. Tables 6 and 7 present comparisons of these 
preferred factors by present influence category for professional school faculty 
‘in college and‘university settings, respectively. (Readers who are interested 

- tn — such Comparisons for faculty in other disciplines should consult the 


pfeviously cited Final Report.) Examination of Tables 6 and 7 indicates that 
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Table 5 


Factors Faculty Are Uncertain About Regarding 
Their Influence in P/T Decisions* 


FPQ Factor . Settings in which 

Label and Number ; uncertainty'' lie 

Colleges Universities 
4 Support of Department Policy and Goals v v 
7 Formal Colleague Appraisals/Teaching v 
8 Grant Support for Research ¥ v 
G Supervision of Theses : v 
10 Personality Traits and Attitudes v 
11 Consultation Record on/off Campus v 


12 Service on Institution/System Committees 
13 Academic Advising 

14 Membership in Professional Organizations 
15 Service (no fee) to Community 

16 Supervision of Field Placements 

17 Elected Offices in Organizations ™~ 
18 Public Presentations of Products 

19 Informal Colleague Appraisals/Research 
20 Formal Colleague Appraisals/Research v 
21 Obtaining Advance Degree / 


SSS SS RS 


23 Informal Colleague Appraisals/Service v 

25 Evidence of Student Learning in Courses v 

26 Election to Faculty Senate v 

27 Effort to Remain Current in Discipline v v 

28 Credit Hour Production v 

29 Student Demand for Course v y 

31 Evaluation by School/Department Committee v 

32 Availability to Students ¥ v 

33 Innovative Effort in Teaching v yj 
34 Formal Peer Appraisals (from off campus) y 7 


“Ambiguous P/T factors have been operationally defined as those FPQ Factors 
with means ranging from 1 to 3.2 and certainty scores < 30 for col Leyes 
and < 25 for universities. 
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Table 6 
A Comparison of Preferred Factors by Present 


Influence Category for Professional Schools: 
College Settings - (N=54) 


SOS 


ae Percent 
Grouping Preferred Indicating, Present 
Rank "Preferred Rank 


Preferred and presently influential: 


Evaluation by Department Chairman 4 35% 2 
Personality Traits and Attitudes 4 35% 4 


Student Ratings 5 33% 5 


Preferred and presently ambiguous: 


Effort to Remain Current in Discipline | 4hy >10 
Innovative Effort in. Teaching 2 39% >10 
Evidence of Student Learning in Courses 4 35% >10 


Preferred and presently uninfluential: 


(None) 


Presently influential but not. preferred 


by 30% or more: 


‘Obtaining Advance Degree * l 
Time in Academic Rank 3 
Formal Colleague Appraisals/Teaching 6 
Service on Departmental Committees 7 
Formal Peer Appraisals (from off campus) 8 
Publications . 9 


Table 7 
A Comparison of Preferred Factors by Present 


Influence Category for Professional Schools: 
University Settings - (N=54) 


Percent 
Grouping Preferred Indicating Present 
. Rank Preferred’! Rank 


Preferred and presently influential: 
Publications 


Student Ratings 


Preferred and presently ambiguous: 


(None) 


Preferred and presently uninfluential: 


(None) 
“Presently influential put not preférred 
by 30% or more: 
Evaluation by Department Chairman 
Evaluation by School/Department Committee 
Time in Academic Rank 


Public Presentations of Products 


Informal Colleague Appraisals/Teaching 


» 


Service on Departmental Committees 


Service on Institution/System Committees 
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neither group of professional school faculty perceived any ''preferred'' factors 
as being presently uninfluential. However, data from professional school faculty 
in college settings indicated that three factors (viz., effort to remain current 
in discipliné, innovative effort in teaching, and evidence of student learning 
in courses) are presently ambiguous and preferred. Student ratings of instructor 
performance was a preferred factor for professional school faculty in both college 
and university settings. The only other factor that was preferred by professional 
school faculty in university settings was publications in scholarly journals and 
books. Professional school faculty in college settings also preferred evaluations 
by department chairman and personality traits and attitudes. 

ein passing, it should be pointed out that the respondents selected the five 
factors that they would most prefer to be judged by from the list of 34 FPQ factors. 
Since the list contained a variety of factors for each of the several professorial 
functions, some of which overlapped, the percent indicating ''preferred' should be 
interpreted in light of the difficulty of obtaining matches when seheaiting only 
5 froma list of 34. Regardless of atedenie discipline, university faculty 
when asked for preferences on what factors shou!d be used in the faculty 
evaluation process, most frequently nominated "publications in scholarly journals 
and books'' as a first choice, while the college faculty tended to rahe "evaluations 
by department chairmen." This is consistent with. the general trend that we 
observed on the campuses in the dtraceton of teaching and adnintsteative faculty 
in university settings preferring a more formal product-oriented assessment 
process versus a more informal personality-oriented process that is evident in 
responses Feed college-level faculty. The interview data also reflected these 
contrasts. , 
Implications | 

In general, our findings regarding what influences faculty evaluation decisions 


are consistent with those of Astin and Lee (1966) and Seldin (1975), on colieges and 


offer unique: findings relative to university settings. 
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In both college and university settings three distinct groups of e 
r] 

promotion/tenure factors emerge: influential, uninfluential, and ambiguous. 

The ambiguous group of .factors is, without a doubt, unnecessarily large. The 

size of this cluster probably results in much counterproductive performance by 

faculty who are eee to be all things to all people. 

Our findings suggest to us some alternative courses of actions which may 
be worth administrative and faculty consideration. Some action-oriented 
alternatives are as/follows: : 

| 

1. Increase the degree of ''shared understanding’! among all ranks of 
faculty with regard to performance criteria and institutional 
expectations velatted to each area of arateenertnt functioning. 
Direct action could be taken to establish whether or not these 
ambiguous factors are, or should be, influential or uninfluential 
in the faculty evaluation process. : 

2. Develop and disseminate guifelines, standards, and processes which 
will result in reducing any discrepancies between presently influential 
factors and current institutional faculty evaluation policies and 
criteria. 

3. Develop a means for periodically examining the criteria and supportive 
data used to assess professorial performance to assure more adequate 
communications of expectations to faculty. There is an unmistakable 
lack of supportive data for many of the FPQ factors. This is a par- 
ticularly troublesome problem. Undoubvediv, many of the FPQ factors are 
presently uninfluential, in the eyes of faculty and administrators, 
because of the Fadct that no viable means exist for evaluating some of 
these specific attributes of performance. However, the fact that some 


of these same factors are included in the preferred group makes 


developmental efforts worthy of serious consideration, 
. 
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4, Intensify Petia aimed at developing indicators for assessing evidence 
of student learning, effort to remain current in discipline, and 
~innoyative effort in teaching. Each of these factors tend to be preferred 
by both college and university faculty. - 

The present findings, as a whole, point to significant gaps between institu-— 
tional policy statements regarding promotion/tenure criteria and the data 
collected to verify their attainment. Such findings are not unique to Oregon. 
Other research (e.g., Eble, 1972) covering the last decade has shown faculty 
members to be unable to generate much substantive progress in developing 
equitable and adeqnate methods for assessing performance, whether of faculty, 
administrators, programs or institutions. The reality is that faculty members 
remain in control of what is valued, and of the determination of what evidence 
will be used to substantiate the attainment of those values. Campus administra- 
tors are coimonly torn between this expression of the idealism of the academy on 
the one bein -atie the management realities they must face, including the pressures 
of judging one's colleagues and their work, on the other. The consequence is 
frequently a stalemate regarding proposals for change:in the assessment process. 
However, present natidnal economic forecasts may exert new kinds of pressure 
for change among weatanlea, Campuses are unlikely to be growing in the 
foreseeable future, and therefore it is likely that harsh consteairite snail bear 
upon administrators to assure high quality performance with existing resources. 
The times, for Waites or worse, may be optimal for raising fundamental questions 
about the purposes and methods of evaluating faculty. 

In the short span of our study we have seen some noteworthy efforts to 
improve faculty performance review procedures. Almost without exception, the 
impetus for these efforts originated from highly placed administrators, an 
observation clearly in support of our bias that direction for improvement must, 


comé from the top. This is not to say that we advocate unilateral and/or > 


ak° 


arbitrary imposition of changes from above, To the contrary, we firmly 
believe that faculty members must be active in the design and dev :opment of 
evaluative instruments and procedures. 

In addition to the suggested action-oriented alternatives previously 
presented, where might a campus administrator begin to develop faculty evaluation 
euekiae that would be capable of positively influencing the quality of instruction, 
research, and acl We would recommend a methodology similar to the one used 
ta thts project. A ddta collection effort might be a necessary component of 
such an approach but this decision would have to be made locally based on 
examination of the available data and on local circumstances. for certain, 
two kinds of focused activities should be initiated. The first would be to find 
some ineans for reviewing and thoroughly analyzing institutional policy and 
guideline statements governing salary, tenure and promotion decisions to establish: 
their compliance with administrative rules; that the emphasis given to the three 
major professorial functions is consistent with the recognized mission of the 
institution; the extent to which criteria and their minimum Aseeusnty evidence 
are specified; and the extent go which subjective and objective evidence to be 
gathered by systematic methods is made available to the decision makers. 

The second set of activities should focus upon faculty eprempeinna (and 
preferences) pleardbeg what influences present decisions. The discrepancies 
between presently influential and preferred factors, and there will be many, are’ 
not only informative but also constitute excellent subject matter~for discussions. 
One of the major focal points for these discussions should be clarification regard- 
ing the roles of the various sources of evidence. A likely consequence of such 
dialogue, relative to each-of the various FPQ Sectors. would be either to increase 
or to eliminate types and/or sources of evidence. 


it is our expectation that any campus completing these two sets of activities 


will be in a position to simplify and improve their faculty evaluation orocedures, 
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and thereby provide considerably more realistic incentives for faculty. There 
Y 


should be no justification for either a faculty member pursuing or an administrator 
recommending a "shotgun" approach to either performance or assessment. The 
business of collecting any and all kinds of data having some face validity can, 

and should cease. Our campus-specific findings have repeatedly shown that a very 
limited number of criteria and review sources shou!d satisfy (a) the reality of 
making decisions (decision makers do not utilize dozens of variables in making 

a choice, even though they may think they do), and,(b) the preferences of most 
faculty, who have indicated, by their. responses, that they would prefer to be 
judged with systematic and/or product-oriented evidence. 

A good starting point for simplifying the review process would be to reaffirm 
the use of those criteria and review sources in which there is consensus that 
they are (a) definitely influential and (b) consistent with institutional policy. 
The process should concurrently serve to eliminate al] factors that occupy an 
ambiguous status at present. Al] sek eb¥eay ta should be removed from guideline 
wesienwate in an overt fashion until consensus can be achieved among al} 
connected with the process regarding their value, and operational definitions 
assigned to them, 

As a result of this process we would anticipate that somewhere between four 
and six criteria would eventually emerge and be justified for general use on 
either a college or a university campus. These would definitely include 
(a) student ratings of instructional performance, (b) assessments of publications 
and/or artistic products, and (c) evidence of student learning in courses. Two 
primary review sources are likely: department chairpersons and department or 
school committees. Reviews by interdisciplinary groups (e.g., campus-wide 
promotion/tenure committees) raised serious questions for us because of the wide Re 
disciplinary differences in criterla and standards delineated by our data. Rather 


than force a dilution by merging these discipline criteria, we would prefer to 
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see, and believe thet sufficient justification can be made for, the establishment 
of broad campus procedures and guidelines that would allow disciplines to set°at 


least a major portion of their own criteria. ‘ 


“‘ 
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Table -8 


Spy Sertainty Scores for College Faculty 
Ranks Combined 


by Discipline: All 


The certainty scores listed below may be used to gain some sense of how consensus 


can vary by setting ad each factor. 


Certainty scores range between 0 and 50. 


A 


low ''certainty score’ (near zero) indicates that factor's use is clouded by uncer- 


tainty; conversely, a high score (above 35 or 40) indicates consensus among re- 
spendents regarding the factor's use or non-use. ; 


¢ 


; . Factor ‘Label 
’ ‘ 


Disciplines 


. : ON Humanities Sciences Sciences Schgols 


’ 
WON DUM OWN — 


Publications 


“Student Ratings 
-Service,an bepartmental Committees 


Support Sf Department Palicy and Goals 
Colleague Assessments of Syllabi 
Informal Col league» Appraisals/Teaching 
Formal Colleague Appraisals/Teaching 
Grant Support for Research 

Supervision of Theses 

Personality Traits and Attitudes Ff 
Consultation Record on/off Campus 
Service on Institution/System Commi ttees 
Academic Adyising 

Nembership in Professional Organizations 
Service (no fee) to Community : 
Supervision of. Field Placements 

Elected Offices in Organizations 

Public Presentation¢ of Products 
Informal Colleague Appraisals/Research 
Formal Colleague Appraisals/Research 
Obtatning Advance Degree 

Evaluation by Department Chairman “ 
Informal Colleague Appraisals/Service 
Formal. Colleague Appraisals/Service 
Evidence of Student Learning in Courses 
Election to Faculty Senate 

Effort tc Remain Current in Discipline 
Credit Hour Production 


“Student Demand for Course 


Time it Academic Rank 


Evalua.ion by School/Department Committee 


Availability to Studehts, 
Innovative Effort In Teaching : 
Formal Peer Appraisals (from off campus) 


Arts & Physical Social Professional 
29 34 36 24 
34 36 46 4) 
50 3) 32 33 
24 6 5 24 
13 43 36 28 
29 24 4} 32 
29 4] 36 43 
24 27 27 4 
4S 45 4) 37 
40 27 32 kh 
24 20 14 4 
29 29 23 30 
8 13 18 24 
13 29 9 19 
“3 15 23 22 

24 24 14 2 
24 20 5 20 
“13 34 14 7 
8 4 18 6 
24 48 4) 48 
29 43 23 4}. 
29 4| 27 43 
13 | 9 ri 
29 4S 50 46 
13 29 32 2 
13 13 36 7 
8° 20 Q 13 
8 34 18 4 
3 29 9 13 
18: | 32 by 
29 15 0 4 
13% 13° 5 26 
38 27 27 9 
a 48 36 46 


Certainty Scores for University Faculty 
by Discipline: All Ranks Combined 

s 
The certainty scores listed below may be used to gain some sense of how consensus 
can vary by setting for each factor. Certainty scores range between 0 and 50. A 
low ''certainty score'' (near zero) indicates that factor's use is clouded by uncer- 
tainty; conversely, a high score (above 35 or 40) indicates consensus among re- 
spondents regarding the factor's use or non-use. . 


Disciplines 


Factor Label Arts & Physical Social Professional 
Humanities Sciences Sciences Schools 


| Publications 
2 Student Ratings 
3} Service on Departmental Committees 
4 Support of Department Policy and Goals 
5 Colleague Assessments of Syllabi 
6 Informa} Colleague Appraisals/Teaching 
7 Formal Col league Appraisals/Teaching 
8 Grant Support for Research 
3 Supervision of Theses 
10 Personality Traits and Attitudes 
It Consultation Record on/off Campus 
12 Service on institut ion/System Committees 
13 Acadeaic Advising 
t4 Hembership in Professional Organizations 
15 Service (no fee) to Communi ty 
16 Supervision of Field Placements 
17 Elected Offices in Organizations 
16 Public Presentations of Products 
19 Informal Colleague Appraisals/Research 
20 Formal Colleague Appraisals/Research 
2! Obtaining Advance Degree 
22 Evaluation by Department Chairman 
23 Informal Colleague Appraisals/Service 
24 Formal Colleague Appraisals/Service 
25 Evidence of Student Learning in Courses 
26 Election to Faculty Senate 
Effort to Remain Current in Discipline 
Credit Hour Production 
29 Student Demand for Course 
30 Time in Acadenic Rank 
31 Evaluation by School Department Committee 
j2 Availabrlity to Students 
3} Innovative Effort in Teaching 
. 34 Formal Peer Appraisals (from off campus) 


. 
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Table 10 


Mean FPQ Factor Values for College 
Academic Disciplines - 7 point scale 


Overall: Oygrall Disciplines 
- Factor Label Factor andard Arts/ Physical Social Prof: 
. Mean Deviation Humanities Sciences Sciences Schools 
(N=133) (N=19) (N=43) (N=22) (N=545) 

T Publications 2.6 A :}:) R 2. 
Student Ratings 215 
Service on Departmental Committees . 88 
Support of Department Policy and Goals .33 
Colleague Assessments of Syllabi -40 
Informal Colleague Appraisals/Teaching my 
Formal Colleague Appraisals/Teaching 24 
Grant Support for Research .26 
Supervision of Theses 68 
Personality Traits and Attitudes 
Consultation Record on/off Campus 
Service on Institution/System Committees 
Academic Advising 
Membership in Professional Organizations 
Service (no fee) to Community 
Supervision of Field-Placements 
Elected Offices in Organizations 
Public Presentations of Products 
Informal Colleague Appraisals/Research 
Formal Colleague Appraisals/Research 
Obtaining Advance Degree 
Evaluation by Department Chairman 
Informal Colleague Appraisals/Service 
Formal Colleague Appraisals/Service 
Evidence of Student Learning in Courses 
Election to Faculty Senate 
Effort to Remain Current in Discipline 
Credit Hour Production 
Student Demand for Course 
Time in Academic Rank 
Evaluation by School/Department Committee 
Availability to Students 
Innovative Effort in Teaching 
Formal Peer Apnraisals (from off campus) 
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Table 11 


“~ a 
Mean FPQ Factor Values for University aa ” 
a Academic Disciplines - 7 point scale —___—_ 
Overall Overall as Disciplines 
Factor Label ; Factor Standard Arts/ Physical Social Prof. 
: Mean Deviation Humanities Sciences Sciences Schools 
(N=76) (N=99) (N=70) _ (N=54) 


ublications 5.5 2.03 5.2 3 5.7 5 
2, Student Ratings 3.8 2.01 4.5 3.1 3.8 3.9 
,32 Service on Departmental Committees : 2.6 1.77 3.1 2.1 2.5 2.7 
4 Support of Department Policy and Goals 12 1.89 1.3 1.2 1.3 1] 
5 Colleague Assessments of Syllabi 4 1.00 4 4 4 5 
6 Informal Colleague Appraisals/Teaching 257 1.98 2.8 3.1 2.6 2.1 
7 Formal Colleague Appraisals/Teaching 1.1 1.99 1.1 1.3 1.1 9 
8 Grant Support for Research 3.1 2.58 2.5 4.4 2.3 2.1 
9 Supervision of Theses 1.8 1.99 1.7 2.4 1.4 1.3 
10 Personality Traits and Attitudes Zale 2.26 2.2 2.4 2.3 1.9 
1] Consultation Record on/off Campus 1.0 1.74 a) 8 .8 1.9 ; 
12, Service on Institution/System Committees 2.3 1.83 2.7 1.8 2.5 2.3 x 
13 Academic Advising 1.7 1.93 2.0 V2 1.9 1.8 ' 
14 Membership in Professional Organizations 1.2 1.70 1.3 7 1.1 1.9 
15 Service (no fee) to Community 1.9 1.90 1.8 1.0 2.2 2.9 
16 Supervision of Field Placements 25 1.30 4 9 .6 1 
17 Elected Offices in Organizations 1.7 1.85 lz 1.2 1.6 2.7 
18 Public Presentations of Products 3.3 2.27 3.6 2.8 3.2 3.7 
19 Informal Colleague Appraisals/Research 2.6 2.32 2.6 2.8 2.7 1.9 
20 Formal Colleague Appraisals/Research 2.5 2.75 3.1 3.0 2.4 1.1 
21 Obtaining Advance Degree 2.6 3.04 3.1 1.5 2.9 3.4 
22 Evaluatton by Department Chatrman 4. 2.41 4.3 4,2 3.9 4] 
23 Informal Colleague Appraisals/Service 1.4 1.90 1.6 1.1 1.5 31.3 
24 Formal Colleague Appraisals/Service .6 1.46 9 3 ay 6 
25 Evidence of Student Learning in Courses 9 1.77 1.2 .7 .6 9 
26 Election to Faculty Senate -6 1.27 .7 3 .8 25 
27. ~Effort to Remain Current/in Discipline 2.4 2.43 2.2 2.9 2.1 2.3 
28 Credit Hour Production 8 1.69 1.3 4 1.1 4 
29 Student Demand for Course ~ F 1.2 1.91 1.7 Ay) oof 9 
30Time in Academic Rank ‘ 1 3.4 2.34 3.6 3.4 3.6 2.8 
31 Evaluation by School/Department Conn t tee 4.0 AN: 2.84 3.8 4.0 4.7 3.6 
32 Availability to Students / yt” WE Spe 1.9. 9 1.5 1.5 3h 
33 Innovative Effort in’Teaching . ; v.23 2.15 2.6 1.9 2.0- 2.6 
30 34 Formal Peer Appraisals (from off campus) 2.8 2.86 3.2 3.6 2.3 1.3 


